Rotten Apples or Bad Harvest? What We Are Measuring When We Are Measuring Abuse

Internet security and technology policy research regularly uses technical indicators of abuse in order to identify culprits and to tailor mitigation strategies. As a major obstacle, readily available data are often misaligned with actual information needs. They are subject to measurement errors relating to observation, aggregation, attribution, and various sources of heterogeneity. More precise indicators such as size estimates are costly to measure at Internet scale. We address these issues for the case of hosting providers with a statistical model of the abuse data generation process, using phishing sites in hosting networks as a case study. We decompose error sources and then estimate key parameters of the model, controlling for heterogeneity in size and business model. We find that 84 % of the variation in abuse counts across 45,358 hosting providers can be explained with structural factors alone. Informed by the fitted model, we systematically select and enrich a subset of 105 homogeneous “statistical twins” with additional explanatory variables, unreasonable to collect for all hosting providers. We find that abuse is positively associated with the popularity of websites hosted and with the prevalence of popular content management systems. Moreover, hosting providers who charge higher prices (after controlling for level differences between countries) witness less abuse. These factors together explain a further 77 % of the remaining variation, calling into question premature inferences from raw abuse indicators on security efforts of actors, and suggesting the adoption of similar analysis frameworks in all domains where network measurement aims at informing technology policy.